Image type classification using color discreteness features

ABSTRACT

A method and system for classifying images between natural pictures and synthetic graphics is provided. In embodiments of the invention, a picture/graphic classification method and system implements one-dimensional color discreteness features, two-dimensional color discreteness features, or three-dimensional color discreteness features for image classification of natural pictures and synthetic graphics. In another embodiment of the invention, a picture/graphic combination classification method and system implements one-dimensional color discreteness features, two-dimensional color discreteness features, and/or three-dimensional color discreteness features is used to classify images between natural picture and synthetic graphic classes.

BACKGROUND OF THE INVENTION

The invention relates to image processing. It finds particularapplication in conjunction with classification of images between naturalpictures and synthetic graphics, and will be described with particularreference thereto However, it is to be appreciated that the invention isalso amenable to other like applications.

During the past several decades, products and services such as TVs,video monitors, photography, motion pictures, copying devices,magazines, brochures, newspapers, etc. have steadily evolved frommonochrome to color. With the increasing use of color products andservices, there is a growing demand for “brighter” and more “colorful”colors in several applications Due to this growing demand, display andprinting of color imagery that is visually pleasing has become a veryimportant topic. In a typical color copier application, the goal is torender the scanned document in such a way that it is most pleasing tothe user

Natural pictures differ from synthetic graphics in many aspects, both interms of visual perception and image statistics. Synthetic graphics arefeatured with smooth regions separated by sharp edges On the contrary,natural pictures are often noisier and the region boundaries are lessprominent. In processing scanned images, it is sometime beneficial todistinguish images from different origins (e.g., synthetic graphics ornatural pictures), however, the origin or “type” information about ascanned image is usually unavailable. The “type” information isextracted from the scanned image. This “type” information is then usedin further processing of the images. High-level image classification canbe achieved by analysis of low-level image attributes geared for theparticular classes Coloring schemes (e g, gamut-mapping or filteringalgorithms) are tailored for specific types of images to obtain qualityreproduction Once an image has been identified as a synthetic graphicimage, further identification of image characteristics can be used tofine-tune the coloring schemes for more appealing reproductions. Themost prominent characteristics of a synthetic graphic image includepatches or areas of the image with uniform color and areas withuniformly changing colors. These areas of uniformly changing color arecalled sweeps

Picture/graphic classifiers have been developed to differentiate betweena natural picture image and a synthetic graphic image by analyzinglow-level image statistics. For example, U.S. Pat. No. 5,767,978 toRevankar et al discloses an adaptable image segmentation system fordifferentially rendering black and white and/or color images using aplurality of imaging techniques An image is segmented according toclasses of regions that may be rendered according to the same imagingtechniques Image regions may be rendered according to a three-classsystem (such as traditional text, graphic, and picture systems), oraccording to more than three image classes. In addition, two imageclasses may be required to render high quality draft or final outputimages. The image characteristics that may be rendered differently fromclass to class may include half toning, colorization and other imageattributes

Synthetic graphics are typically generated using a limited number ofcolors, usually containing a few areas of uniform colors On the otherhand, natural pictures are more noisy, containing smoothly varyingcolors A picture/graphic classifier can analyze the colors todistinguish between natural picture images and synthetic graphic images

Synthetic graphic images contain several areas of uniform color, linesdrawings, text, and have very sharp, prominent, long edges On the otherhand, natural pictures are very noisy and contain short broken edges Apicture/graphic classifier can analyze statistics based on edges todistinguish between natural picture images and synthetic graphic images.

Classifiers that can be used to solve a certain classification probleminclude statistical, structural, neural networks, fuzzy logic, andmachine learning classifiers. Several of these classifiers are availablein public domain and commercial packages. However, no single classifierseems to be highly successful in dealing with complex real worldproblems Each classifier has its own weaknesses and strengths

The invention contemplates new and improved methods for classifyingimages that overcome the above-referenced problems and others

SUMMARY OF THE INVENTION

A method and system for classifying images between natural pictures andsynthetic graphics is provided. In embodiments of the invention, apicture/graphic classification method and system implementsone-dimensional color discreteness features, two-dimensional colordiscreteness features, or three-dimensional color discreteness featuresfor image classification of natural pictures and synthetic graphics. Inanother embodiment of the invention, a picture/graphic combinationclassification method and system implements one-dimensional colordiscreteness features, two-dimensional color discreteness features,and/or three-dimensional color discreteness features is used to classifyimages between natural picture and synthetic graphic classes.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may take form in various components and arrangements ofcomponents, and in various steps and arrangements of steps The drawingsare for purposes of illustrating embodiments of the invention and arenot to be construed as limiting the invention.

FIG. 1 is a flowchart of an image classification process using SGLDtexture features in an embodiment of the invention;

FIG. 2 is a more detailed flowchart of the SGLD matrix initializationand construction process portion of the flowchart in FIG. 1;

FIG. 3 is a flowchart of an image classification process usingone-dimensional (1-D) color discreteness features in an embodiment ofthe invention,

FIG. 4 is a flowchart of an image classification process usingtwo-dimensional (2-D) color discreteness features in an embodiment ofthe invention,

FIG. 5 is a flowchart of an image classification process usingthree-dimensional (3-D) color discreteness features in an embodiment ofthe invention;

FIG. 6 is a flowchart of an image classification process using acombination of 1-D color discreteness features, 2-D color discretenessfeatures, and 3-D color discreteness features in an embodiment of theinvention;

FIG. 7 is a flowchart of an image classification process using edgefeatures in an embodiment of the invention;

FIG. 8 is a flowchart of an image classification process using acombination of SGLD texture features, color discreteness features, andedge features in an embodiment of the invention; and,

FIG. 9 is a block diagram of an image processing system using a “binary”image classification process (i.e., classification of images betweennatural picture or synthetic graphic classes).

DETAILED DESCRIPTION OF THE INVENTION

Spatial gray-level dependence (SGLD) techniques for image analysis arewell known. SGLD feature extraction creates a 2-D histogram thatmeasures first and second-order statistics of an image. These featuresare captured in SGLD matrices. This was originally proposed for textureanalysis of multi-level images Additionally, since texture featuresdistinguish natural pictures from synthetic graphics, SGLD techniquescan be applied to picture/graphic classification of images Apicture/graphic classifier can be created with algorithms that analyzethe texture features captured in SGLD matrices. Using the SGLD texturefeatures, the classifier works to determine whether a scanned image is anatural picture or synthetic graphic. Furthermore, in color images, theluminance component typically contains enough information to determinethe origin of the image. Therefore, an SGLD matrix that captures theluminance component of an image and a picture/graphic classifier usingthe luminance component from the matrix in a classification algorithmcan determine whether the image is a natural picture or syntheticgraphic.

With reference to FIG. 1, a flowchart of an image classification process100 using SGLD texture features in an embodiment of the invention isshown Generally, the classification process filters an input image tosmooth out halftones, builds an SGLD matrix from the smoothed image,extracts texture features from the matrix, and performs an algorithm todetermine whether the image is a natural picture or synthetic graphicbased on one or more of the texture features

More specifically, the process 100 begins with an input image 102. Theimage is processed using a low-pass filter 104 (e.g., a W×W averagingfilter) to smooth the luminance component and reduce any halftone noiseThe SGLD matrix is basically a GL×GL 2-D histogram, where GL is thenumber of gray levels (e.g., 256) The SGLD matrix is generated by firstperforming an initialization (e.g, set to zero) 106. Next, the SGLDmatrix is built from the smoothed image 108. The SGLD matrix is a 2-Dhistogram corresponding to certain characteristics of the pixels in theinput image. For each pixel (m, n) in the smoothed image, a neighboringvalue is calculated using the following logic and equations:if|x(m, n+d)−x(m, n)|>|x(m+d, n)−x(m, n)|then y(m, n)=x(m, n+d),otherwise y(m, n)=x(m+d, n),  (1),where x(m, n) is the smoothed pixel value at (m, n), (m, n+d) and (m+d,n) are vertical and horizontal neighbors, respectively, and d is a fixedinteger (typically one or two).

With reference to FIG. 2, a flowchart of an embodiment of the SGLDmatrix initialization and construction process is shown. Theinitialization step 106 sets the SGLD matrix to zero and sets a pixelcounter (N) to zero 154 The SGLD matrix is constructed from a low-passfiltered image 152 provided by the low-pass filter 104 Construction ofthe SGLD matrix begins by getting a pixel (m, n) 156 from the filteredimage. A neighboring value for the pixel (m, n) is calculated using thealgorithm in equation (1). If |x(m, n+d)−x(m, n)|>|x(m+d, n)−x(m, n)|158, then y(m, n)=x(m, n+d) 160. Otherwise, y(m, n)=x(m+d, n) 162 As isapparent, if pixel (m, n) is in a flat area where x(m, n) is equal toy(m, n), the entry [x(m, n), y(m, n)] is on the diagonal. On the otherhand, if (m, n) is on an edge, the difference between x(m, n) and y(m,n) will be significant, and [x(m, n), y(m, n)] will be far away from thediagonal.

The entry [x(m, n), y(m, n)] in the SGLD matrix is then increased by oneand the pixel counter (N) is increased by one. Next, a check is made todetermine if the calculation was for the last pixel 166 of the inputimage If so, SGLD matrix construction is complete and the SGLD matrix isready for feature extraction 168. Otherwise, the next pixel is retrieved156 from the input image

For the matrix, the neighboring pixels in synthetic graphic images areexpected to be either correlated or very different In other words, forsynthetic graphic images, SGLD matrix entries are usually either on thediagonal or far away from the diagonal. This is because most pixels areeither at the flat regions or on the edges. On the other hand, pixels ofnatural pictures are not expected to have many abrupt changes.Accordingly, masses are expected to be concentrated at the entries thatare near the diagonal for natural picture images. This shows the noisynature of the natural picture images.

Returning to FIG. 1, many features (e.g., variance, bias, skewness,fitness) can be extracted from the SGLD matrix to classify the inputimage between natural picture and synthetic graphic classes The featurescan be implemented individually or combined in various methods (e g,linear combination) Once the SGLD matrix is built, a feature orcombination of features is selected for extraction 110 and processedusing feature algorithms For example, a first feature algorithm measuresvariance (V) (i.e., the second-order moment around the diagonal) 112 andis defined as:V=Σ _(|n−m|>Δ) s(m, n) (m−n)² /N  (2),where s(m, n) is the (m, n)-th entry of the SGLD matrix, Δ is an integerparameter typically between 1 and 16 and,N=Σ _(|n−m|<Δ) s(m, n)  (3)

As the summation is over all (m, n) such that |m−n|>Δ, all the pixels inthe flat regions are ignored. For synthetic graphic images, theremaining pixels are on the edges, while for natural picture images,both pixels in the noisy regions and pixels on the edges are includedVariance (V) is typically larger for synthetic graphic images than fornatural picture images

The second feature algorithm measures average bias (B) 114 and isdefined asB=Σ _(|n−m|<Δ) s(m, n)[n−μ(m)]² /N  (4),where μ(m) is the mean of s(m, n) for a fixed m. For a given m, thedistribution of s(m, n) is roughly symmetrical about the diagonal fornatural picture images, as noise typically has a zero mean symmetricaldistribution As a result B is usually small for natural picture imagesFor synthetic graphic images, s(m, n) is usually unsymmetrical and B islarge

The third feature algorithm measures skewness (S) 116 and is defined as:$\begin{matrix}{{S = {{skewness} = {\sum\limits_{n = 0}^{{GL} - 1}{\frac{{{\sum\limits_{m = 0}^{{GL} - 1}{{{n - m}}\left( {n - m} \right){s\left( {m,n} \right)}}}}^{\frac{1}{2}}}{\sum\limits_{m = 0}^{{GL} - 1}{{{n - m}}{s\left( {m,n} \right)}}}{{c(n)}/C}}}}},{{where}:}} & (5) \\{{c(n)} = {{\sum\limits_{m = 0}^{{GL} - 1}{{s\left( {m,n} \right)}\mspace{14mu}{and}\mspace{14mu} C}} = {\sum\limits_{n = 0}^{{GL} - 1}{{c(n)}.}}}} & (6)\end{matrix}$

The fourth feature algorithm measures fitness (F) 118 and is defined tobe $\begin{matrix}{{F = {{fitness} = \frac{\sum\limits_{n = 0}^{{GL} - 1}{\left( {n - m} \right)^{2}{s\left( {m,n} \right)}}}{\sigma^{2}}}},} & (7)\end{matrix}$where σ is defined such that $\begin{matrix}{{\sum\limits_{d = 0}^{\sigma}\left\lbrack {{s\left( {m,{m + d}} \right)} + {s\left( {m,{m - d}} \right)}} \right\rbrack} = {0\mspace{11mu} 6 \times C}} & (8)\end{matrix}$

The image type decision 120 compares the result of the featurealgorithm(s) to previously selected low and high thresholds (i.e., T_(L)and T_(H), respectively) depending on the algorithm(s) and combinationsselected. If the result of the feature algorithm(s) is below the lowthreshold (T_(L)), the image is classified as a natural picture 122. Ifthe result exceeds the high threshold (T_(H)), the classification issynthetic graphic 126. Obviously, if the behavior of a particularfeature is converse to this logic, the decision logic can be easilyreversed to accommodate If the result of the feature algorithm(s) isequal to or between the low and high thresholds, the class of the imagecannot be determined (i e, indeterminate 124) from the feature orcombination of features selected. It is understood that a number ofother alternatives are possible. For example, a result equal to aparticular threshold can be said to be determinative of the image class,rather than indeterminate Also, in certain circumstances the low andhigh threshold can be equal.

With reference to FIG. 3, a flowchart of an image classification process200 using 1-D color discreteness features in an embodiment of theinvention is shown. The process 200 begins with an input image 202.First, the input image is transformed into a color space 204, in whichthe classification is performed. Although CIELUV space is used in theembodiment being described, many other color spaces can also be used.Next, the image is smoothed using an averaging filter 206 to remove anynoise due to halftones For example, a 4×4 filter was used successfullyNext, a 1-D color histogram is computed for a color channel (i e.,luminance (L), U, and V) 208, any combination of two 1-D colorhistograms may be computed, or all three 1-D color histograms may becomputed, depending on the algorithm or combination of algorithms to beperformed for classification.

Next, the histogram(s) may be pre-filtered using a low-pass filter, forexample an averaging filter 209 to smooth out artificially boostedprobabilities due to the size of the sample. The averaging filter, forexample, works particularly well for images with a relatively smallsample size that lead to a sparse distribution over the histogram. Next,the histogram(s) may be pre-adjusted by applying an F(H) function tohistogram entries with high bin counts representing large areas 210 inthe image in the same color. Predetermined thresholds define high bincounts and determine whether or not the F(H) function is applied to thehistogram bin The F(H) function is a monotonically decreasing function(e g, cube root) and under-weighs the histogram counts for the largeareas. This under-weighing, for example, works particularly well tocorrect a bias to incorrectly classify pictures with large flat regions(e.g., sky, paper background) as synthetic graphic images.

The L, U, and/or V histograms are normalized 211 by the number of pixelsin the image. The color representation scheme is invariant underrotation and translation of the input image and the normalizationprovides scale invariance If I(i) is the histogram of an image, wherethe index i represents a histogram bin, then the normalized histogram His defined as follows $\begin{matrix}{{H(i)} = \frac{I(i)}{\sum\limits_{i = 0}^{{GL} - 1}{I(i)}}} & (9)\end{matrix}$

Since synthetic graphics are generated using a limited number of colors,synthetic graphic images usually are comprised of a few areas of uniformcolor Hence, the color histograms for a synthetic graphic image usuallycontain several sharp peaks. On the other hand, natural pictures usuallycontain more colors with smoothly varying transitions. Hence, naturalpictures are more noisy and produce histograms containing fewer andsmoother peaks. This difference in the histograms is captured in colordiscreteness algorithms for each color channel (i.e., R_(—)L algorithm212, R_(—)U algorithm 214, and R_(—)V algorithm 216) The colordiscreteness algorithms are defined as follows. $\begin{matrix}{{{R\_ L} = {\sum\limits_{x = 1}^{{GL} - 1}{{{{H\_ L}(x)} - {{H\_ L}\left( {x - 1} \right)}}}}},} & (10) \\{{{R\_ U} = {\sum\limits_{x = 1}^{{GL} - 1}{{{{H\_ U}(x)} - {{H\_ U}\left( {x - 1} \right)}}}}},} & (11) \\{{{R\_ V} = {\sum\limits_{x = 1}^{{GL} - 1}{{{{H\_ V}(x)} - {{H\_ V}\left( {x - 1} \right)}}}}},} & (12)\end{matrix}$where GL is the number of bins in the H_(—)L, H_(—)U, and H_(—)V colorhistograms. The algorithms in equations (10)–(12) are derived from the1-D color discreteness algorithm for any color space, defined for colorchannel s of a generic color space as follows:R _(s) =Σ|H _(s)(x)−H _(s)(x−1)|  (13)

The image type decision 218 compares the results of the 1-D colordiscreteness algorithms (i.e., 212, 214, or 216) selected forperformance to previously selected thresholds (e.g,., low threshold(T_(L)) and high threshold (T_(H))) If the result of the selected 1-Dcolor discreteness algorithm is above T_(H) or below T_(L), the image isclassified as either a synthetic graphic 126 or natural picture 122according to predetermined rules. Otherwise, the class of the imagecannot be determined (i.e, indeterminate 124) by the 1-D colordiscreteness feature (i e, R_(—)L, R_(—)U, or R_(—)V) Alternatively, theclassifier may use all three 1-D color discreteness features in anysequence, any combination of two features in any sequence, or any onefeature (as described above).

With reference to FIG, 4, a flowchart of an image classification process300 using 2-D color discreteness features in an embodiment of theinvention is shown Like the process 200 using 1-D color discretenessfeatures, the process 300 begins with an input image 202, the inputimage is transformed into a color space 204, and the image is smoothedusing an averaging filter 206 Again, although CIELUV space is used inthe embodiment being described, many other color spaces can also beused. Next, a 2-D color histogram is computed for two color channels(i.e., LU, LV, or UV) 308, any combination of two 2-D color histogramsmay be computed, or all three 2-D color histograms may be computed,depending on the algorithm or combination of algorithms to be performedfor classification

Next, like the process 200 using 1-D color discreteness features, the2-D histogram(s) may be pre-filtered using an averaging filter 209 andmay be pre-adjusted by applying an F(H) function to histogram entriesrepresenting large areas 210. Pre-filtering is important for the 2-Dhistogram(s), as they are typically sparse. Next, the LU, LV, and/or UVhistograms are normalized 311 by the number of pixels in the image usingequation (9)

Next, like the process 200 using 1-D color discreteness features, thedifference between histograms of natural picture images and syntheticgraphic images is captured in 2-D color discreteness algorithms (i.e.,R_(—)LU algorithm 312, R_(—)LV algorithm 314, and R_(—)UV algorithm316). The 2-D color discreteness algorithms for the LUV color space aredefined as follows: $\begin{matrix}{{{R\_ LU} = {\sum\limits_{x = 1}^{{GL} - 1}{\sum\limits_{v = 1}^{{GL} - 1}\begin{pmatrix}{{{{{H\_ LU}\left( {x,y} \right)} - {{H\_ LU}\left( {{x - 1},y} \right)}}} +} \\{{{{{H\_ LU}\left( {x,y} \right)} - {{H\_ LU}\left( {x,{y - 1}} \right)}}}\mspace{20mu}}\end{pmatrix}}}},} & (14) \\{{{R\_ LV} = {\sum\limits_{x = 1}^{{GL} - 1}{\sum\limits_{v = 1}^{{GL} - 1}\begin{pmatrix}{{{{{H\_ LV}\left( {x,y} \right)} - {{H\_ LV}\left( {{x - 1},y} \right)}}} +} \\{{{{{H\_ LV}\left( {x,y} \right)} - {{H\_ LV}\left( {x,{y - 1}} \right)}}}\mspace{20mu}}\end{pmatrix}}}},} & (15) \\{{{R\_ UV} = {\sum\limits_{x = 1}^{{GL} - 1}{\sum\limits_{y = 1}^{{GL} - 1}\begin{pmatrix}{{{{{H\_ UV}\left( {x,y} \right)} - {{H\_ UV}\left( {{x - 1},y} \right)}}} +} \\{{{{{H\_ UV}\left( {x,y} \right)} - {{H\_ UV}\left( {x,{y - 1}} \right)}}}\mspace{20mu}}\end{pmatrix}}}},} & (16)\end{matrix}$where GL is the number of bins for each component in the H_(—)LU,H_(—)LV, and H_(—)UV color histograms. The algorithms in equations(14)–(16) are derived from the 2-D color discreteness algorithm for anycolor space, defined for color channel s and color channel t of ageneric color space as followsR _(st)=Σ(|H _(st)(x,y)−H_(st)(x−1,y)|+|H _(st)(x,y)−H_(st)(x,y−1)|)  (17)

The image type decision 318 compares the results of the 2-D colordiscreteness algorithm (i.e., 312, 314, or 316) selected for performanceto previously selected thresholds (e.g., low threshold (T_(L)) and highthreshold (T_(H))). If the result of a 2-D color discreteness algorithmis above T_(H) or below T_(L), the image is classified as either asynthetic graphic 126 or natural picture 122 according to predeterminedrules. Otherwise, the class of the image cannot be determined (i.e.,indeterminate 124) by the 2-D color discreteness feature (i.e., R_(—)LU,R_(—)LV, or R_(—)UV). Alternatively, the classifier may use all three2-D color discreteness features in any sequence, any combination of twofeatures in any sequence, or any one feature (as described above).

With reference to FIG. 5, a flowchart of an image classification process400 using 3-D color discreteness features in an embodiment of theinvention is shown Like the process 200 using 1-D color discretenessfeatures and the process 300 using 2-D color discreteness features, theprocess 400 begins with an input image 202, the input image istransformed into a color space 204, and the image is smoothed using anaveraging filter 206. Again, although CIELUV space is used in theembodiment being described, many other color spaces can also be usedNext, a 3-D color histogram is computed for the three color channels(i.e., LUV) 408.

Next, like the process 200 using 1-D color discreteness features and theprocess 300 using 2-D color discreteness features, the 3-D histogram maybe pre-filtered using an averaging filter 209 and may be pre-adjusted byapplying an F(H) function to histogram entries representing large areas210 Pre-filtering is important for the 3-D histogram, as it is typicallysparse Next, the LUV histogram is normalized 411 by the number of pixelsin the image using equation (9)

Next, like the process 200 using 1-D color discreteness features and theprocess 300 using 2-D color discreteness features, the differencebetween histograms of natural picture images and synthetic graphicimages is captured in a 3-D color discreteness algorithm (i.e., R_(—)LUValgorithm 414) The 3-D color discreteness algorithm is defined asfollows $\begin{matrix}{{{R\_ LUV} = {\sum\limits_{x = 1}^{{GL} - 1}{\sum\limits_{y = 1}^{{GL} - 1}{\sum\limits_{z = 1}^{{GL} - 1}\begin{pmatrix}{{{{{H\_ LUV}\left( {x,y,z} \right)} - {{H\_ LUV}\left( {{x - 1},y,z} \right)}}} +} \\{{{{{{H\_ LUV}\left( {x,y,z} \right)} - {{H\_ LU}\left( {x,{y - 1},z} \right)}}} +}\mspace{20mu}} \\{{{{{H\_ LUV}\left( {x,y,z} \right)} - {{H\_ LU}\left( {x,y,{z - 1}} \right)}}}\mspace{40mu}}\end{pmatrix}}}}},} & (18)\end{matrix}$where GL is the number of bins for each component in the H_(—)LUV colorhistogram. The algorithm in equation (18) is derived from the 3-D colordiscreteness algorithm for any color space, defined for a generic colorspace as follows $\begin{matrix}{R = {\sum\begin{pmatrix}{{{{H\left( {x,y,z} \right)} - {H\left( {{x - 1},y,z} \right)}}} +} \\{{{{H\left( {x,y,z} \right)} - {H\left( {x,{y - 1},z} \right)}}} +} \\{{{{H\left( {x,y,z} \right)} - {H\left( {x,y,{z - 1}} \right)}}}\mspace{20mu}}\end{pmatrix}}} & (19)\end{matrix}$

The image type decision 418 compares the results of the 3-D colordiscreteness algorithm (i.e, 414) to previously selected thresholds (eg, low threshold (T_(L)) and high threshold (T_(H))) If the result of a3-D color discreteness algorithm is above T_(H) or below T_(L), theimage is classified as either a synthetic graphic 126 or natural picture122 according to predetermined rules. Otherwise, the class of the imagecannot be determined (i.e., indeterminate 124) by the 3-D colordiscreteness feature (i.e., R_(—)LUV).

With reference to FIG. 6, a flowchart of an image classification process500 using a combination of 1-D color discreteness features, 2-D colordiscreteness features, and 3-D color discreteness features in anembodiment of the invention is shown Notably, this color discretenesscombination classifier includes features of the three classifiers using1-D color discreteness features, 2-D color discreteness features, and3-D color discreteness features discussed above By combining the threecolor discreteness classifiers, performance may be improved over anindividual color discreteness classifier. The color discretenesscombination classifier can take advantage of reduced processing overheadby making an early classification, for example, using the 1-D colordiscreteness features where the classification is more obvious.Conversely, the color discreteness combination classifier can takeadvantage of increased precision by continuing to analyze the statisticsof the input image using 2-D color discreteness features and 3-D colordiscreteness features.

Like the individual processes 200, 300, 400, the process 500 begins withan input image 202, the input image is transformed into a color space204, and the image is smoothed using an averaging filter 206. Again,although CIELUV space is used in the embodiment being described, manyother color spaces can also be used.

The process 500 continues by performing steps 208 through 216 of the 1-Dcolor discreteness classifier 508 shown in FIG. 3 The image typedecision 510 compares the results of the 1-D color discretenessalgorithms (i.e., 212, 214, or 216) selected for performance topreviously selected thresholds (e.g., low threshold (T_(L)) and highthreshold (T_(H))). If the result of the selected 1-D color discretenessalgorithm is above T_(H) or below T_(L), the image is classified aseither a synthetic graphic 126 or natural picture 122 according topredetermined rules Otherwise, the class of the image cannot bedetermined (i.e, indeterminate 512) by the 1-D color discretenessfeature (i e., R_(—)L, R_(—)U, or R_(—)V) Alternatively, the 1-D colordiscreteness classifier may use all three 1-D color discretenessfeatures in any sequence, any combination of two features in anysequence, or any one feature during step 508.

If the result of the 1-D color discreteness classifier is indeterminate512, the process 500 continues by performing steps 308 through 316 ofthe 2-D color discreteness classifier 514 shown in FIG. 4 The image typedecision 516 compares the results of the 2-D color discretenessalgorithms (i e, 312, 314, or 316) selected for performance topreviously selected thresholds (e.g. low threshold (T_(L)) and highthreshold (T_(H))). If the result of a 2-D color discreteness algorithmis above T_(H) or below T_(L), the image is classified as either asynthetic graphic 126 or natural picture 122 according to predeterminedrules Otherwise, the class of the image cannot be determined (i.e,indeterminate 518) by the 2-D color discreteness feature (i e., R_(—)LU,R_(—)LV, or R_(—)UV) Alternatively, the 2-D color discretenessclassifier may use all three 2-D color discreteness features in anysequence, any combination of two features in any sequence, or any onefeature during step 514

If the result of the 2-D color discreteness classifier is indeterminate518, the process 500 continues by performing steps 408 through 414 ofthe 3-D color discreteness classifier 520 shown in FIG. 5 The image typedecision 522 compares the results of the 3-D color discretenessalgorithm (i.e., 414) to previously selected thresholds (e.g., lowthreshold (T_(L)) and high threshold (T_(H))). If the result of a 3-Dcolor discreteness algorithm is above T_(H) or below T_(L), the image isclassified as either a synthetic graphic 126 or natural picture 122according to predetermined rules. Otherwise, the class of the imagecannot be determined (i.e., indeterminate 124) by the 3-D colordiscreteness feature (i.e., R_(—)LUV).

Alternatively, the color discreteness combination classifier may performthe 1-D color discreteness classifier, the 2-D color discretenessclassifier, and 3-D color discreteness classifier in any sequence andmay perform all three color discreteness classifiers or any two colordiscreteness classifiers before making an image type decision

With reference to FIG. 7, a flowchart of an image classification process600 using edge features in an embodiment of the invention is shown Theprocess 600 begins with an input image 602. First, edges of color areasin the image are detected 604 using a standard Canny edge detector andan edge map image is created. The parameters identified for the edgedetector were determined empirically Deviations that produce suitableresults are also contemplated Next, the edges in the edge map image areconnected 606 (e g., using a standard 8-connected component algorithm).The average number of pixels per connected edge (P/E) in the edge mapimage is used as a feature 608. The algorithm for this edge feature isdefined as $\begin{matrix}{{P/E} = \frac{{{No}.\mspace{14mu}{of}}\mspace{14mu}{Edge}\mspace{14mu}{Pixels}}{{{No}.\mspace{14mu}{of}}\mspace{14mu}{Connected}\mspace{14mu}{Edges}}} & (20)\end{matrix}$

Typically, synthetic graphic images have fewer connected edges, but eachconnected edge includes a large number of pixels On the other hand,pictures have a lot more connected edges, but usually very few pixels ineach connected edge. This feature is particularly accurate for highvalues In other words, if the value of P/E is high, it is almost certainthat the image is synthetic graphic image. However, if the value of P/Eis low, the classification of the image is less certain (e.g.,indeterminate). This is because the P/E value may be low for syntheticgraphic images that have low frequency halftones or certain types ofbackgrounds Accordingly, the image type decision 610 compares the resultof the P/E feature algorithm to a previously selected high threshold (ie., T_(H)) If the result exceeds the high threshold (T_(H)), theclassification is synthetic graphic 126. Otherwise, the class of theimage cannot be determined (i.e., indeterminate 124).

The orientation of edges (EO) in the edge map image is also used as afeature 609. As shown in FIG. 7, the connected edges 606 in the edge mapimage are used to compute an edge orientation histogram 607 For example,the Hough Transform may be used to compute the edge orientationhistogram The Hough Transform is a well-known method for finding lines.A detailed description of the Hough Transform can be found in “DigitalPicture Processing”, by Azriel Rosenfeld and Avinash C. Kak, (AcademicPress, Inc, 1982) Vol. 2, pp 121–126 The Hough Transform converts anedge map image into a 2-D histogram with one dimension being the lineorientation and the other being the line intercept A Hough Transformentry HT (x, y) represents the length of a line that has an orientationof x and an intercept of y. The edge orientation histogram H(x) 607 canbe obtained by manipulating the HT (x,y) histogram as follows:$\begin{matrix}{{{H(x)} = \left( {\sum{{HT}\left( {x,y} \right)}^{2}} \right)^{\frac{1}{2}}},} & (21)\end{matrix}$where the summation is over all y values.

The edge orientation (EO) algorithm 609 is performed on the edgeorientation histogram H(x) 607 as followsEO=−ΣH(x)log H(x)  (22).

It has been observed that synthetic graphic images often contain manylong straight edges that are oriented in the same direction, typicallyhorizontal and vertical directions, while the edges in natural pictureimages are typically shorter and oriented more randomly. If many of theedge pixels have a common orientation, the resulting histogram will showone or more spikes The result of the edge orientation algorithm (i.eentropy measure) for a spiky histogram is a low number. Accordingly, theimage type decision 612 compares the result of the EO feature algorithm609 to a previously selected low threshold (i e., T_(L)) If the resultis less than the low threshold (T_(L)), the classification is syntheticgraphic 126 Otherwise, the class of the image cannot be determined(i.e., indeterminate 124)

Furthermore, it is understood that additional edge features may be usedfor classification of images between natural picture and syntheticgraphic classes. Any combination of edge features can also be used bythe classifier in any sequence.

With reference to FIG. 8, a flowchart of an image classification process700 using a combination of SGLD texture features, color discretenessfeatures, and edge features in an embodiment of the invention is shown.Notably, this image classifier combines all the features of the threetypes of classifiers discussed above, including 1-D, 2-D, and 3-Dfeatures in regard to color discreteness. By combining SGLD texture,color discreteness, and/or edge features in one classifier, performancemay be improved over classifiers using a single feature.

The process 700 begins with an input image 702. Next, the features areextracted from the input image 704 Feature extraction includes compilingSGLD texture features 706 (e.g., variance (V), bias (B), skewness (S),fitness (F)), color discreteness features 708, including 1-D colordiscreteness features, (e g, R_(—)L, R_(—)U, R_(—)V), 2-D colordiscreteness features, (e g, R_(—)LU, R_(—)LV, R_(—)UV), and 3-D colordiscreteness features, (e g, R_(—)LUV), and edge features 710 (e g.,P/E, EO).

The SGLD texture features are extracted by performing steps 104–110 ofthe process 100 depicted in FIG. 1. Similarly, the 1-D colordiscreteness features are extracted by performing steps 204–211 of theprocess 200 depicted in FIG. 3. Likewise, the 2-D color discretenessfeatures are extracted by performing steps 204–311 of the process 300depicted in FIG. 4 Similarly, the 3-D color discreteness features areextracted by performing steps 204–411 of the process 400 depicted inFIG. 5. Likewise, the edge features are extracted by performing steps604–607 of process 600 depicted in FIG. 7.

Next, the feature algorithms are performed 716 The SGLD texture featurealgorithms are performed by accomplishing steps 112–118 of the process100 depicted in FIG. 1 Similarly, the 1-D color discreteness featurealgorithms are performed by accomplishing steps 212–216 of the process200 depicted in FIG. 3 Likewise, the 2-D color discreteness featurealgorithms are performed by accomplishing steps 312–316 of the process300 depicted in FIG. 4. Similarly, the 3-D color discreteness featurealgorithm is performed by accomplishing step 414 of the process 400depicted in FIG. 5 Likewise, the edge feature algorithms are performedby accomplishing steps 608 and 609 of the process 600 depicted in FIG.7.

Finally, the image type decision 718 compares the results of the featurealgorithms (e.g., V, B, S, F, R_(—)L, R_(—)U, R_(—)V, R_(—)LU, R_(—)LV,R_(—)UV, R_(—)LUV, P/E, EO) to previously selected thresholds (e.g., lowthresholds (T_(L)) and/or high thresholds (T_(H))) associated with eachfeature. If the result of any feature algorithm is above the associatedT_(H) or below the associated T_(L), the image is classified as either asynthetic graphic 126 or natural picture 122 according to predeterminedrules Otherwise, the class of the image cannot be determined (i.e.,indeterminate 124) by the combination classifier. The combinationclassifier may also incorporate logic to resolve conflicting results,particularly where one individual classifier is indeterminate 124 andanother individual classifier classifies the image as either a syntheticgraphic 126 or natural picture 122

The embodiment described and shown in FIG. 8 uses all features availableto the combination classifier and performs feature extraction 704 in aparallel fashion, performs feature algorithms 716 in a parallel fashion,and then makes a combination image type decision 718. Alternatively, thecombination classifier may use all features available to it in anysequence In other words, feature extraction 704, feature algorithms 716,and image type decisions 718 can be performed for individual features,in a serial fashion, similar to that described above for the colordiscreteness combination classifier shown in FIG. 6 Furtheralternatives, using a serial-parallel combination in any order with anycombination of any two or more features in parallel legs of such acombination classifier are also contemplated. Many other alternativecombination classifiers are also available by using any combination oftwo or more of the features available to the combination classifier inparallel fashion, serial fashion, or serial-parallel fashion

With reference to FIG. 9, a block diagram of an image segmentationsystem 800 using a “binary” image classification process (i e,classification of images between natural picture or synthetic graphicclasses) is shown. The picture/graphic classifiers (i.e., 100, 200, 300,400, 500, 600, 700) of FIGS. 1–8 are “binary” classifiers and could beimplemented in such a system 800 As described above for FIGS. 1–8, aninput image 802 is provided to a feature extractor 804. The featureextractor 804 extracts pertinent characteristics (i e, features) basedon the parameters required by algorithms of the binary classifier 806.The binary classifier 806 exercises algorithms designed to classify theinput image between a natural picture or a synthetic graphic image(e.g., [0, 1] where 0 indicates natural picture and 1 indicatessynthetic graphic). This binary classification result is provided to aswitch 808. The switch 808 receives the input image 802 and switches itbetween natural picture processing 810 and synthetic graphic processing812, depending on the binary classification result

Natural picture processing 810 processes the image in a manner tailoredto maximize the quality of natural picture images (e g, gamut mapping)Similarly, synthetic graphic processing 812 is tailored to maximizes thequality of synthetic graphic images (e.g., filtering). If the inputimage is classified as a picture, the input image 802 is switched tonatural picture processing 810 and a picture output 814 is produced.Alternatively, if the image is classified as a synthetic graphic, theinput image 802 is switched to synthetic graphic processing 812 and asynthetic graphic output 816 is produced. In the event that the binaryclassifier 806 cannot determine the class of the input image, one of theprocesses (e.g., natural picture processing 810) may be selected bydefault.

The invention has been described above with reference to variousembodiments Obviously, modifications and alterations will occur toothers upon reading and understanding the preceding detailed descriptionIt is intended that the invention be construed as including all suchmodifications and alterations insofar as they come within the scope ofthe appended claims or the equivalents thereof

1. A method for classification of an input image in picture or graphicclasses, comprising the following steps: a) extracting one or moreone-dimensional color discreteness features from the input image; b)conditioning each extracted feature to prepare the feature forsubsequent processing; c) processing each conditioned feature using analgorithm associated with the feature; d) comparing the result of eachfeature algorithm to one or more previously selected thresholds; and e)if, according to previously determined rules, any comparison isdeterminative of the class of the input image, classifying the inputimage in either the picture or graphic classes according to thepreviously determined rules, otherwise indicating the result isindeterminate.
 2. The method as set forth in claim 1, wherein step a)includes the following steps: f) transforming the input image to a colorspace; g) processing the transformed image using a low-pass filter; andh) computing one or more one-dimensional histograms from the filteredimage, each histogram representing a different color channel of thecolor space.
 3. The method as set forth in claim 2, wherein step b)includes the following step: i) normalizing the one or moreone-dimensional histograms to condition the histogram(s) for subsequentprocessing.
 4. The method as set forth in claim 3, wherein step b)further includes the following step: j) processing the one or moreone-dimensional histograms using a low-pass filter to condition thehistogram(s) for subsequent processing.
 5. The method as set forth inclaim 3, wherein step b) further includes the following step: j)processing each bin of the one or more one-dimensional histograms wherethe bin count exceeds a previously selected threshold using a previouslyselected mathematical function to condition the histogram(s) forsubsequent processing.
 6. The method as set forth in claim 2, whereinthe color space is a CIELUV color space.
 7. The method as set forth inclaim 6, wherein the one or more one-dimensional histograms are selectedfrom the group consisting of an L channel histogram, a U channelhistogram, and a V channel histogram.
 8. A method for classification ofan input image in picture or graphic classes, comprising the followingsteps: a) extracting one or more two-dimensional color discretenessfeatures from the input image; b) conditioning each extracted feature toprepare the feature for subsequent processing; c) processing eachconditioned feature using an algorithm associated with the feature; d)comparing the result of each feature algorithm to one or more previouslyselected thresholds; and e) if, according to previously determinedrules, any comparison is determinative of the class of the input image,classifying the input image in either the picture or graphic classesaccording to the previously determined rules, otherwise indicating theresult is indeterminate.
 9. The method as set forth in claim 8, whereinstep a) includes the following steps: f) transforming the input image toa color space; g) processing the transformed image using a low-passfilter; and h) computing one or more two-dimensional histograms from thefiltered image, each dimension of the histogram representing a differentcolor channel of the color space.
 10. The method as set forth in claim9, wherein step b) includes the following step: i) normalizing the oneor more two-dimensional histograms to condition the histogram(s) forsubsequent processing.
 11. The method as set forth in claim 10, whereinstep b) further includes the following step: j) processing the one ormore two-dimensional histograms using a low-pass filter to condition thehistogram(s) for subsequent processing.
 12. The method as set forth inclaim 10, wherein step b) further includes the following step: j)processing each bin of the one or more two-dimensional histograms wherethe bin count exceeds a previously selected threshold using a previouslyselected mathematical function to condition the histogram(s) forsubsequent processing.
 13. The method as set forth in claim 9, whereinthe color space is a CIELUV color space.
 14. The method as set forth inclaim 13, wherein the one or more two-dimensional histograms areselected from the group consisting of an LU histogram, an LV histogram,and a UV histogram.
 15. A method for classification of an input image inpicture or graphic classes, comprising the following steps: a)extracting a three-dimensional color discreteness feature from the inputimage; b) conditioning the extracted feature to prepare the feature forsubsequent processing; c) processing the conditioned feature using analgorithm associated with the feature; d) comparing the result of thefeature algorithm to one or more previously selected thresholds; and e)if, according to previously determined rules, any comparison isdeterminative of the class of the input image, classifying the inputimage in either the picture or graphic classes according to thepreviously determined rules, otherwise indicating the result isindeterminate.
 16. The method as set forth in claim 15, wherein step a)includes the following steps: f) transforming the input image to a colorspace; g) processing the transformed image using a low-pass filter; andh) computing a three-dimensional histogram from the filtered image, eachdimension of the histogram representing a different color channel of thecolor space.
 17. The method as set forth in claim 16, wherein step b)includes the following step: i) normalizing the three-dimensionalhistogram to condition the histogram for subsequent processing.
 18. Themethod as set forth in claim 17, wherein step b) further includes thefollowing step: j) processing the three-dimensional histogram using alow-pass filter to condition the histogram for subsequent processing.19. The method as set forth in claim 17, wherein step b) furtherincludes the following step: j) processing each bin of thethree-dimensional histogram where the bin count exceeds a previouslyselected threshold using a previously selected mathematical function tocondition the histogram for subsequent processing.
 20. The method as setforth in claim 16, wherein the color space is a CIELUV color space. 21.The method as set forth in claim 20, wherein the three-dimensionalhistogram is an LUV histogram.